Unit 28: Recommender Systems + Course Recap

Introduction

In this final lecture, we explore Recommender Systems — one of the most impactful applications of machine learning in everyday life. From Netflix movie suggestions to Amazon product recommendations, these systems drive engagement and revenue across digital platforms. We examine two main approaches: content-based filtering and collaborative filtering (user-based and item-based). We then conclude with a comprehensive recap of the entire course, mapping out the machine learning landscape we've traversed together.

Learning Objectives:

Theory

1. The Long Tail Phenomenon

Physical stores face scarcity of resources — limited shelf space means they can only stock popular items. Online platforms, however, can make everything available. This distinction is known as the Long Tail.

Items (ranked by popularity) Popularity Head (Popular items) Long Tail (Niche items) Physical stores can only offer this Online stores can offer everything

Figure: The Long Tail distribution. Physical retailers focus on the "head" (popular items), while online platforms can serve the entire tail (niche items).

2. Types of Recommender Systems

Type Approach Example
Content-Based Analyze item properties to recommend similar items Recommend action movies to users who watched action movies
Collaborative Filtering (User-Based) Find similar users and recommend what they liked Users who bought X also bought Y
Collaborative Filtering (Item-Based) Find similar items to those the user liked and recommend them Because you watched Movie A, you might like Movie B

3. User-Based Collaborative Filtering

In user-based CF, we define similarity between two users as the correlation between their ratings on items they've both rated. If one user rates a movie higher than average, it's likely the similar user will too.

Pearson Correlation Similarity between users u and v:
$$sim(u,v) = \frac{\sum_{i \in I_{uv}}(r_{ui} - \bar{r}_u)(r_{vi} - \bar{r}_v)}{\sqrt{\sum_{i \in I_{uv}}(r_{ui} - \bar{r}_u)^2} \sqrt{\sum_{i \in I_{uv}}(r_{vi} - \bar{r}_v)^2}}$$
Predicted rating for user u on item i:
$$\hat{r}_{ui} = \frac{\sum_{v \in N_u} sim(u,v) \cdot r_{vi}}{\sum_{v \in N_u} |sim(u,v)|}$$

4. Item-Based Collaborative Filtering

In item-based CF, we first determine how similar items are based on user ratings, then predict ratings by giving more weight to ratings of items most similar to the target item.

Cosine Similarity between items i and j:
$$sim(i,j) = \frac{\vec{r}_i \cdot \vec{r}_j}{||\vec{r}_i|| \cdot ||\vec{r}_j||} = \frac{\sum_{u} r_{ui} \cdot r_{uj}}{\sqrt{\sum_{u} r_{ui}^2} \sqrt{\sum_{u} r_{uj}^2}}$$
Predicted rating for user u on item i:
$$\hat{r}_{ui} = \frac{\sum_{j \in N_i} sim(i,j) \cdot r_{uj}}{\sum_{j \in N_i} |sim(i,j)|}$$

5. User-Item Rating Matrix Example

Consider the following ratings matrix (1-5 scale, ? = missing):

User \ Movie M1 M2 M3 M4
U153?1
U24?52
U31135
U42345

6. Course Roadmap Recap

Classification
Regression
Clustering
Other Topics

Classification Methods

Regression Methods

Clustering Methods

Evaluation: Elbow Method, Silhouette Coefficient, Intrinsic vs. Extrinsic metrics

Other Key Topics

7. ML Project Cycle

Raw Data Collection Dataset Training Dataset Test Dataset Processed Training Data ML Algorithm Predictive Model Final Model Iterate & Evaluate Evaluate Preprocessing 1:
Missing data,
Feature extraction Preprocessing 2:
Scaling,
Dimensionality reduction New Dataset + Pipeline Apply

Figure: The Machine Learning Project Cycle from data collection to deployment.

Interactive Examples

Collaborative Filtering Similarity Calculator

Compute cosine similarity between two items based on user ratings:

User Item A Rating Item B Rating
U1
U2
U3
U4

Rating Prediction Calculator

Predict U1's rating for M3 using user-based CF:

Similar UserSimilarity to U1Rating for M3
U20.9655
U30.4233
U40.6584

User-Based vs. Item-Based Comparison

Aspect User-Based CF Item-Based CF
Similarity computed between Users Items
Best when # users < # items # items < # users
Stability Less stable (user preferences change) More stable (item similarity is static)
Sparsity handling Poor with sparse data Better with sparse data
Interpretability "Users like you enjoyed..." "Because you watched X..."

Numerical Solutions

Problem 1: User-Based CF Rating Prediction

Given: Predict U1's rating for M3.

User \ MovieM1M2M3M4
U153?1
U24?52
U31135
U42345

User Similarities to U1: U2=0.965, U3=0.423, U4=0.658

Step-by-Step Solution

Step 1: Identify relevant users

Users U2, U3, and U4 have all rated M3. Their similarities to U1 are 0.965, 0.423, and 0.658 respectively.

Step 2: Apply prediction formula

$$\hat{r}_{U1,M3} = \frac{sim(U1,U2) \cdot r_{U2,M3} + sim(U1,U3) \cdot r_{U3,M3} + sim(U1,U4) \cdot r_{U4,M3}}{sim(U1,U2) + sim(U1,U3) + sim(U1,U4)}$$

Step 3: Substitute values

$$\hat{r}_{U1,M3} = \frac{0.965 \times 5 + 0.423 \times 3 + 0.658 \times 4}{0.965 + 0.423 + 0.658}$$ $$= \frac{4.825 + 1.269 + 2.632}{2.046} = \frac{8.726}{2.046} \approx 4.26$$

Step 4: Interpret

The predicted rating is approximately 4.3 (or 4 when rounded to integer). Since U2 is most similar to U1 and gave M3 a high rating (5), the prediction is pulled toward the higher end.

Problem 2: Item-Based CF with Cosine Similarity

Given: Transposed matrix for item-based CF.

MovieU1U2U3U4
M15412
M23None13
M3None534
M41255

Task: Compute cosine similarity between M1 and M2 using shared users U1, U3, U4.

Step-by-Step Solution

Step 1: Extract rating vectors

For shared users U1, U3, U4:

  • M1 vector: [5, 1, 2]
  • M2 vector: [3, 1, 3]

Step 2: Compute dot product

$$\vec{r}_{M1} \cdot \vec{r}_{M2} = 5(3) + 1(1) + 2(3) = 15 + 1 + 6 = 22$$

Step 3: Compute magnitudes

$$||\vec{r}_{M1}|| = \sqrt{5^2 + 1^2 + 2^2} = \sqrt{25 + 1 + 4} = \sqrt{30} \approx 5.477$$ $$||\vec{r}_{M2}|| = \sqrt{3^2 + 1^2 + 3^2} = \sqrt{9 + 1 + 9} = \sqrt{19} \approx 4.359$$

Step 4: Compute cosine similarity

$$sim(M1,M2) = \frac{22}{5.477 \times 4.359} = \frac{22}{23.87} \approx 0.9215$$

This high similarity (0.9215) indicates M1 and M2 are very similar items.

Problem 3: Item-Based Rating Prediction

Given: Item similarities: M1-M2=0.9215, M1-M3=0.9567, M1-M4=0.5567. U1 rated M2=3, M4=1. Predict U1's rating for M1.

Step-by-Step Solution

Step 1: Identify rated similar items

U1 has rated M2 (sim=0.9215) and M4 (sim=0.5567). M3 was not rated by U1.

Step 2: Apply item-based prediction

$$\hat{r}_{U1,M1} = \frac{sim(M1,M2) \cdot r_{U1,M2} + sim(M1,M4) \cdot r_{U1,M4}}{sim(M1,M2) + sim(M1,M4)}$$ $$= \frac{0.9215 \times 3 + 0.5567 \times 1}{0.9215 + 0.5567}$$ $$= \frac{2.7645 + 0.5567}{1.4782} = \frac{3.3212}{1.4782} \approx 2.25$$

Note: This prediction (2.25) differs from U1's actual rating of 5 for M1, illustrating that item-based CF can produce inaccurate predictions when the user has rated few similar items.

Try-It-Yourself Problems

Problem 1: Pearson Correlation

Compute the Pearson correlation between U1 and U3 using common movies M1, M2, M4:

MovieU1U3
M151
M231
M415

U1 mean = 3, U3 mean = 7/3 ≈ 2.33

Step 1: Compute centered ratings:

MovieU1 - 3U3 - 2.33
M12-1.33
M20-1.33
M4-22.67

Step 2: Numerator = 2(-1.33) + 0(-1.33) + (-2)(2.67) = -2.66 + 0 - 5.34 = -8.0

Step 3: Denominator:

√(4 + 0 + 4) × √(1.77 + 1.77 + 7.13) = √8 × √10.67 = 2.83 × 3.27 = 9.25

Step 4: Correlation = -8.0 / 9.25 ≈ -0.865

Interpretation: Strong negative correlation — U1 and U3 have opposite preferences!

Problem 2: Cold Start Problem

A new user U5 has rated only one movie: M2 = 4. Explain why both user-based and item-based CF struggle with this scenario, and propose a solution.

User-based CF struggles because:

  • We cannot compute reliable similarity with only one common rating.
  • Any similarity computed would be based on a single data point and highly unreliable.

Item-based CF struggles because:

  • We only have one rated item (M2) to base recommendations on.
  • The prediction for any new item depends entirely on M2's similarity to that item.

Solutions:

  1. Content-based filtering: Use movie genres, actors, directors to recommend similar movies to M2.
  2. Hybrid approach: Combine collaborative filtering with demographic information (age, location).
  3. Popular items: Recommend globally popular items until more ratings are collected.
  4. Explicit onboarding: Ask new users to rate a set of diverse items during signup.
Problem 3: Course Concept Map

For each of the following scenarios, identify the most appropriate ML technique from this course:

  1. Predicting whether an email is spam or not spam
  2. Grouping customers into segments based on purchasing behavior
  3. Predicting house prices based on square footage, location, and age
  4. Reducing a dataset from 500 features to 10 while preserving variance
  5. Explaining why a neural network denied a loan application
  1. Classification: Naïve Bayes (text data), Logistic Regression, or Random Forest
  2. Clustering: K-Means (partitional) or Agglomerative Hierarchical Clustering
  3. Regression: Gradient Boosting Regressor or Ridge/Lasso Regression
  4. Dimensionality Reduction: Principal Component Analysis (PCA)
  5. Explainable ML: LIME (Local Interpretable Model-agnostic Explanations)

Interactive Quiz

Question 1: In user-based collaborative filtering, similarity between users is typically computed using:

Euclidean distance between item features
Pearson correlation of ratings on commonly rated items
Manhattan distance between user demographics
Jaccard similarity of purchased item sets

Question 2: The "Long Tail" phenomenon implies that:

Only popular items generate revenue
Online stores can profit from niche items that physical stores cannot stock
Recommendation systems should only recommend popular items
User ratings follow a normal distribution

Question 3: Which clustering algorithm can discover clusters of arbitrary shape and identify noise points?

K-Means
DBSCAN
Agglomerative Clustering
PCA

Question 4: In the ML project cycle, what should be done after training a model candidate?

Immediately deploy to production
Evaluate via cross-validation and iterate if needed
Collect more raw data
Skip testing to save time

Question 5: Which regularization technique performs automatic feature selection?

Ridge Regression
Lasso Regression
Ordinary Least Squares
k-NN Regression

Key Takeaways

Common Pitfalls

Resources